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Air  Force  Relevance 


•  Current  influence  domain  (e.g.,  Influence  Operations,  Joint  Targeting 
Cycle,  Military  Deception,  Psychological  Operations)  must  adapt  to  meet 
the  demands  of  irregular  warfare 

-  “Aone-size-fits-all  deterrence  strategy  will  not  suffice  in  the  future  joint  operating 
environment.  Such  an  approach  assumes  that  we  fully  understand  the  thought 
processes,  strategic  culture,  and  value  hierarchy  and  can  precisely  ascertain  ‘red 
lines’  of  the  enemy”  (Joint  Operating  Environment,  2010;  p.  64). 

•  The  necessary  Precision  infiuence  can  be  realized  through  basic  and 
applied  research  in  the  social  sciences 
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Trust  =  Influence 


•  Developing  and  applying  theoretical-driven  and  empirically  supported 
models  of  trust  is  a  key  leverage  point 

•  Trust  is  a  critical  factor  driving  human  decision  making  and  behavior 

-  Key  determinant  of  influence  (Sweeney  et  al.,  2009) 

-  ...of  automated  tools  use  (Lee  &  See,  2004) 

-  ...of  performance  &  positive  work  attitudes  (Dirks  &  Ferrin,  2002) 

-  . .  .of  ability  to  adapt  (Stokes,  Lyons,  &  Schneider,  2011) 


3 


Distribution  A.  Approved  for  public  release;  distribution  unlimited. 
88ABW-2014-1 127,  cleared  20  March  2014 


The  Trust  Process 


Beliefs  &  Perceptions  Attitudes  Intention  Behavior 


Trustor  Attributes 

-  Cultural  norms 

-  Personal 
relevance 


Intention 


Trustworthiness 


Trustee 
Attributes 
•  Ability 
Benevolence 
Integrity 


Behavior/ 

Decision-making 


I  Intentions  translate  to 
'  behavior  depending  on 
I  environmental  and 
I  cognitive  constraints 
I  e.g.,  Cognitive  load, 

I  situation  a\A/areness,  social 


I 


networks 


Modified  model  based  on  Lee  &  See  (2004),  Mayer  et  al.  (1995),  and 
Colquitt  et  al.  (207. 
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Trustworthiness:  Ability, 
Benevolence,  Integrity 

(ABI) 

*  Parsimonious  foundation  of  trustworthiness  indicators  (Mayer  etai.,  1995) 

-  Ability:  that  group  of  skills,  competencies,  and  characteristics  that  enable  a 
party  to  have  influence  within  some  specific  domain. 

-  Benevolence:  extent  to  which  a  trustee  is  believed  to  want  to  do  good  to  the 
trustor,  aside  from  an  egocentric  profit  motive. 

-  Integrity:  perception  that  the  trustee  adheres  to  a  set  of  principles  that  the 
trustor  finds  acceptable. 

•  To  date,  there  have  been  no  such  studies  that  have  empirically  tested  this 
model  in  different  cultures. 
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Cultural  Norms 


Foundational  psychological  processes  are  dependent,  in  part,  on  the 
surrounding  culture  within  which  individuals  live  (Fiske,  Kitayama,  Markus,  & 
Nisbett,  1998) 


Cuiture  has  been  defined  according  to  several  dimensions  which  are 
beiieved  to  influence  the  mechanisms  through  which  peopie  evaluate  self 
and  others,  situations,  and  relationships  (Fiske  et  al.,  1998;  Hofstede,  1980; 
Huff  &  Kelley,  2003) 

-  Individualism-Collectivism 

-  Power  Distance 

-  Masculinity-Femininity 

-  Uncertainty  Avoidance 
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Individualism-Collectivism 


•  Individualism:  Characterized  by  strive  for  independent  success,  formulate 
and  evaluate  self  based  on  personal  goals,  make  independent  choices, 
and  base  decisions  on  utility 
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Individualism- 
Collectivism  Continued.... 


Coilectivism:  characterized  by  value  of  the  group  above  self,  emphasize 
relationships  and  group  goais,  motivated  to  maintain  harmony,  and 
evaiuate  life  based  on  collective  needs 


—  Greater  value  for  relationships 

and  social  harmony  (Fiske  et  al.,  1998) 

—  Asian  managers  use  more 
relationship-oriented  appeals  as 
influence  tactic  (Fu  &  YukI,  2000) 


HI:  Participants  from  an  collectivistic  culture  will  rate  trust 
higher  when  trustee  benevolence  is  high 
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Cognitive  Load  (CL) 


CL  represents  the  ioad  that  performing  a  particuiar  task  imposes  on  the 
cognitive  system  and  is  a  key  component  of  human  information 
processing  (Paas  et  al.,  1994;  Parasuraman  et  al.,  2000) 

increased  CL  ieads  to  entrenchment  of  estabiished  behaviors  in  reiation  to 
user  interfaces  (Oviatt  et  al.,  2004) 

The  Affect  Infusion  Model  (Forgas,  1995)  suggests  that  during  faster 
processing,  individuals  use  their  affective  states  as  a  short-cut  to  infer 
their  evaluative  reactions  to  a  target 

-  Similar  process  may  occur  with  reliance  on  cultural  norms  as  a  heuristic 
during  high  CL 
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Personal  Relevance 


Relative  impact  of  ABI  indicators  may  vary  according  to  the 
type  of  activity  to  be  performed 

•  Action-oriented  (direct  involvement) 

—  Ability  strongest  trust  predictor  for  situations  where  teams  worked  on  a  joint 
project  (Serva  et  al.,  2005) 


H4:  Participants  will  select  applicants  with  higher  ability  in  direct 
_ involvement  situations  (e.g.,  direct  supervisor) _ 


•  Judgment-oriented 

—  Integrity  strongest  trust  predictor  in  politically  sensitive  situations,  e.g.,  performance 
appraisal  system  (Mayer  &  Davis,  1999;  Mayer  et  al.,  1995) 


H5:  Participants  will  select  applicants  with  higher  integrity  in 
_ judgment-oriented  situations  (e.g.,  non-profit  org) 
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Personal  Relevance 
Continued.... 


•  Relationship-oriented  (limited  empirical  evidence) 


H6:  Participants  will  select  applicants  with  higher  benevolence  In 
_ relationship-oriented  situations  (e.g.,  co-worker) _ 
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Method 


•  Design:  3  (culture)  x  2  (cog.  load)  x  3  (ABI),  mixed 

•  Target  N  =  120  per  country  (U.S.,  Malaysia,  Australia) 

-  U.S.  sample  classified  as  individualistic,  Malaysian  as  collectivistic,  consistent 
with  previous  taxonomies  (Fiske  et  al.,  1998) 

-  Australian  sample  will  serve  as  a  comparison  group  for  the  more  traditional 
individualistic  and  collectivistic  nations 

-  Participants  recruited  from  local  universities 

•  Measures 

-  Audio  recording 

-  Trust  Propensity  (8-item:  Mayer  &  Davis,  1999) 

•  “One  should  be  very  cautious  with  strangers.” 
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Method  Continued 


-  Trust  (10-item;  Mayer  &  Davis,  1999) 

•  “If  I  had  my  way,  I  wouldn't  let  this  person  have  any  influence  over  issues  that  are 
important  to  me.”* 

-  CL  (manipulation  check) 

•  “Please  rate  your  mental  effort  in  the  task  you  just  performed.” 
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Task  1  -  Trust  Rating 


^  A^pl.i&arLt  59-Q  Appl j-Ea±i^  Ap^lxc-ajit  4"^  S  Applic-aiit  2  ~  | 


Applicant  59— Q 


E3Epeir±enc:e 


Project  Manager /Consult  ant  at  Bing 
an<l  Jalcobson^  Inc. 


Education 


EarlbecJ  University,  1996 

Bac^fofor  of  Arts:  Business  Finance 


\  "Is  kind  and  caring  toward  others.  Goes  well  above  and 
,4—^  beyond  the  call  of  duty  to  look  outforother  students'  best 
interests."  -  Professor  of  Business  Finance 


Bing  and  Jakobson,  Inc.,  2003  -  Present 
Rroject  /l^na^&r  /  Consuftant 

•  Responsibilities:  D ev elop ed  fo rmal  p resentatio ns  fo r  potential 

customers,  the  delivery  of  a  final  product  to  customers  under  strict 
deadlines,  and  development  of  team  building  exercises. 


"Puts  forth  an  extra  effort  to  support  others.  Takes  good  care  of  the 
employees."  -  Executive  Supervisor  , 


Creations  4  Us,  1  996  -  2003 
Human  Services  il^anacfer 

Responsibilities:  Organization  and  development  of  train!? 
customer  care  relations 


3  rog  rams  fo r 


VolunteeiT  Escperxencje  and  Personal  Interests 


Community  Dancing  Club,  1999  -  Rresent 
lUfember 

•  Assisted  in  the  organization  and  planning  of  dancing  lessons  and  events. 


ABI  Manipulations 
(benevolence) 


"Bends  over  backwards  to  help  you  out  whenever  possible.  Has  always  been  particularly  good  to  me."  -  Co-Club  Mg 


The  appl i c-aTit  WDuld  he  very  c.apah-le  c-f  per f ormin-g- 
hi s /he  r  j  ch , 

mii'  applicant  woulsl  he  aucce  s  s  at  the  thin-ga  he /she 

^riea  to  _ 

The  applicant  would  have  knowledge  ah-out  the  work  that 
needs  done . 

3r  i^eel  very  confident  ahout  the  applicant's  skd’lTsr 


Stron  g  ly 
Disagree 

0 

Di  s  ag  ree 

0 

Somev/hat 

Disagree 

0 

Neither  Agree 
or  Disagree 

0 

Somev/hat 

Agree 

0 

Agree 

0 

Strong  ly  Ag  ree 

0 

a  II 

1 

II 

vp 

0 

0 

0 

0 

0 

0 

0 

II 

••Q 

II 

Q 

Q 
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Benevolence 


Task  2  -  Job  Selection 


L  Fill  Positions 


Rani  Applicants 


Applicamt  5  9— Q 

•^"Is  kind  and  caning;  toward  others..  Goes  well  above  and 
jeyond  the  call  of  duty  to  look  out  for  other  students'"  best 
nterests.".  Professor  of  Business  Finance 

^"Futs  forth  an  extra  effort  to  support  others.  Takes  good 
isre  of  the  employees.".  Executive  Supem  isor 

3ends  over  backwards  to  help  you  out  whenever  possible. 
iHas  always  been  particularly  good  to  me.",  Co-Club  ]Meniber 


A^pILicant  47  — S 


‘'^"Consistently  demonstrated  strong  skills  on  class  projects 
■and  understanding  of  course  material.".  Professor  of 
iManagemeut 

r'^"A  good  employee  who"  s  technical  know-how  is  widely 
regarded  as  state-of-the-art.".  Executive  Super\vsor 

I '^"Proficient  and  competent  while  completing  work. 
iPerforms  quality  work.  Always  gets  things  done  well.",  Co- 

!Club  ^lember 


^  Applx-cairt  68— KL 

"Acts  on  the  up  and  up  from  what  Tve  seen  and  heard. 
Always  fair  and  honest.".  Professor  of  Business  Finauce 

‘■^"Has  a  strong  moral  foundation.  Always  does  the  right 
thing  even  in  morally  difficult  situations.".  Executive 
Supervisor 

honest,  just,  and  impartial  \\T.thout  fail.",  Co-Club 
^Member 


JikppILCcajrit  24— V 


''^"Judgment  is  not  always  ethically  sound.".  Professor  of 
Business 

'■^"D-oes  not  have  a  strong  set  of  principles.  Lacks  a  moral 
center.".  Executive  Supervisor 

'■^"We  have  known  each  other  for  about  two  years.", 

Co-Club  Member 


Drag  and  Drop  the 
applicant  to  a  position. 

Please  select  the  level  of 
confidence  you  have  in 
your  decision. 


-litoiLiir  Supe  i^vxsoir : - 


Drop  Applicuant  Hoto 


r  Un  s  e 


pCo- Worker? :  - 


Drop  Applicant  Here 


[Unselecteci 


i  Ottieir  '  s  Su.pervi.sor  : 


Dirop  Applioant  He  ire 


I  Uixs  e  leo  fceci 


Con-tirLue 
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Task  3  -  Ranking 


^  FiJ-l  Poait-j-ons  RarLk  Applic-ant-s 


Af>pILi_ciaj.T.t  5  9  — Q 


•^"Ib  kind  and  caring  toward  others.  Gk>es  well  above  and 
beyond  the  call  of  duly  to  look  out  for  other  students"  best 
interests.".  Professor  of  Business  Finance 

'"^"Puts  forth  an  extra  effort  to  support  others.  Takes  good 
care  of  the  employees.".  Executive  Supetrisor 

"Bends  over  backwards  Eo  help  you  out  whenever  possible. 
jHas  always  been  particularly  good  to  me.",  Co-Ctub  ^lember 


^  ApiplxGa3.i.t  47  — S 


•^"Consistently  demonstrated  strong  skills  on  class  projects 
and  understanding  of  course  material.".  Professor  of 
M  anagem  eut 

"A  good  employee  who"  s  technical  know-how  is  w'idely 
regarded  as  state-of-the-art.".  Executive  Superx'isor 


"Proficient  and  competent  while  completing  work. 
Performs  q-uality  work.  Always  gets  things  done  well.". 

Club  ^lember 


Co- 


^  68— K1 


■jf^"Acts  on  the  up  and  up  from  what  Tve  seen  and  heard. 
xAlw^ays  fair  and  honest.".  Professor  of  Bus  Lues  s  Fiuauce 

■•^"Has  a  strong  moral  foundation.  Always  does  the  right 
thing  even  in  morally  difficult  situations.".  Executive 
Supervisor 

"Is  honest,  just,  and  impartial  w'ithout  fail.",  Co-Club 
^lember 


^  J4p'plxGar:it  24— V 


'^"Judgment  is  not  always  ethically  sound.".  Professor  of 
Bus  Lues  s 

’"^"D-oes  not  have  a  strong  set  of  principles.  Lacks  a  moral 
center.".  Executive  Supervisor 

■^"We  have  known  each  other  for  about  tv^^o  years.", 

Co-Club  ^Member 


Please  rank  all  applicant 
from  best  (1)  to  worst  (4) 
choice  for  each  position. 


C3J  Dr-op  Applic-ant  Horo 
(4)  D'rop  Applic-an.'b  Ho  re 


Continue 
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High  CL 


59 -Q  j^pj -■  cant  6b-K.  Apj^l  i-C-ant  47 -S  i-£>l3--~^an^~  24-^' 


A.ppl±oa.ii'b  59  — Q 


fm 


E3Epear±enc:e 


Project  Manager /Consult  ant  at  Bing^ 
and  Jakobson^  Xnc . 


Education 


EarlbecJ  University,  1996 

Bac^sfor  of  Arts:  Business  Finance 


"Is  kind  and  caring  toward  others.  Goes  well  above  and 
beyond  the  call  of  duty  to  look  outforother  students'  best 
interests."  -  Professor  of  Business  Finance 


Bing  and  Jakobson,  Inc.,  2003  -  Present 
ProJ&ct  Mana^&r  /  Consu/tant 

•  Responsibilities:  D ev  elop ed  fo rmal  p resentatio ns  fo r  potential 

customers,  the  delivery  of  a  final  product  to  customers  under  strict 
deadlines,  and  development  of  team  building  exercises. 

Jf  "Puts  forth  an  extra  effort  to  support  others.  Takes  good  care  of  the 
employees."  -  Executive  Supervisor 


Creations  4  Us,  1  996  -  2003 
Human  Services  Mana^&r 

•  Responsibilities:  Organi 

customer  care  relations 


VolunteeiT  E3cpeir±enc:e  a.ndL  Peirsonal  Inteirests 

Community  Dancing  Club,  1999  -  Present 
ilfemher 

•  Assisted  in  the  organization  and  planning  of  dancing  lessons  and  events. 

"Bends  over  backwards  to  help  you  out  whenever  possible.  Has  always  been  particularly  good  to 


TSSK  Ust 

X 

rams  fo  r 

Pllonly 

»ppliciiht^'P 

ME[c> 

Apphcant 

HtOH 

Strongly 

Disagree  Disagree 

The  appl i eaTit:.  WQ-uld  he  very  capable  c-f  perforTniiig  ^  ^ 

hi  3  /he  r  j  c-h .  ^  ^ 

Somewhat 

Disagree 

o 

Hefther 

or  Dis 

c 

The  applicant:  WQTJ.lii  he  anccessf-ul  at  the  things  he /she 
trie  s  to  do  _ 

Q 

Q 

The  applicant  wotild  have  knowledge  ahont  the  work  that  ^  ^ 

needs  done  .  ^  ^ 

o 

c 

T.  feel  very  confident  ahiout  the  applicant's  skills. 

O; 

;Q 

Q; 

ngty  Agr&e 

o 


Applicant  5A-C  (LOW^ 

for 

Res  y  m  e  Va  I  idator  TN 


Add  , 

T 
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Preliminary  Results 
Job  Selection 

•  CL  manipulation  successfui 

-  High  M:  3.625,  Low  M:  3.037;  f(72)=5.201,  p<.001 

•  ABI  indicators  predicting  Position  Seiection 

-  Appears  ABI  model  only  predictive  under  low  CL 


LowCL  Condition 

%  of  time  selected  for  position  of 

.. 

Applicant 

Supervisor 

Other's  Supervisor 

Co-Worker 

HiAbility 

0.250 

0.175 

0.575 

HiBenev 

0.513 

0.238 

0.250 

Hilntegrity 

0.238 

0.575 

0.175 

Neutral 

0.000 

0.013 

0.000 

Total 

1.000 

1.000 

1.000 

HiCL  Condition 

%  of  time  selected  for  position  of 

.. 

Applicant 

Supervisor 

Other's  Supervisor 

Co-Worker 

HiAbility 

0.250 

0.275 

0.450 

HiBenev 

0.350 

0.325 

0.313 

Hilntegrity 

0.400 

0.388 

0.200 

Neutral 

0.000 

0.013 

0.038 

Total 

1.000 

1.000 

1.000 
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Preliminary  Results 
Trust  Ratings 


Low  CL 
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Preliminary  Interpretation 


•  ABI  model  did  not  hold  under  high  CL  conditions  for  personal  relevance 


-  Trust  judgments  based  more  on  Trustor  attributes  and  heuristic  processing 

-  Rational  models  of  cognition  and  judgment  (such  as  ABI  model)  fail  to  capture 
complex  and  dynamic  nature  of  human  choice  and  behavior  in  “the  real  world” 
where  cognitive  demands  are  high  (Kahneman,  2003;  Klein,  1993) 

-  Kahneman  distinguishes  b/w  : 

•  Intuitive  reasoning  -  rapid,  automatic,  affect-laden  (used  for  High  CL) 

•  Objective  reasoning  -  slow,  effortful,  evaluative  (used  for  Low  CL) 
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Preliminary  Interpretation 
Continued . 


•  Next  Steps  w/  remaining  data  collection: 

-  Refine  current  analyses  as  needed  and  conduct  primary  hypothesis  tests 

-  Examine  state  affect  and  personality  effects  on  trust  judgments  and  position 
selection 

-  Examine  if  cognitive  effects  hold  in  other  cultures 

-  Compare  position  selections  based  on  ABI  in  other  cultures 

-  Ability  approached  significance  under  high  CL;  will  another  ABI  indicator  be 


significant  (or  close)  in  a  collectivist  culture? 
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Dynamic  Model  of  Trust 


1.  Effects  of  CL 

Grant  AOARD-1 1-4056 


Charlene  Stokes,  Joseph  Lyons,  Kevin  Eschelman,  Johanna  Culbertson 
US  AFRL,  Dayton 
Mei  Hua  Lin 


Applicant  35-tJ 


trOl|(Kil  HjadKiL'-r'  it  Dlslil,  Ijiu. 

DhC1.ri.  Inc.,  .3103' 
r'rviatlh^im^ut 

•■  Bf^pqnU^Wi^f:  Supqnitfd 

*  mdddl  rqr  dnvfapdd*  td  ffUd-ld  p*rfdiiii»id»  ^ 

H  Educa-kicn 

thiUiErn  IJntvBniCr-'n99 

,  Is  ivalU'ibh'StnfihijfHiAhii  p«ffoiTiK3 4t -a  htgh  l»«l  '  'Eiaculiva 

4'^  SuiitiTiMr 

’^Pirfitni*!)'  dM4nn«l4d  tlidna  Mlt  dn.  dbM  pidiMI^ 

J(.rn«Cnn-^5,  1»$.  J»3 

^  ^  i>H{l4Ttl^ai9  df  mJdiiil  AdldddddrdT 

'  BtrtindH 

-  fmspqn^^iWif^  «iiSl^w>q  pipjfdH-  pn  (f mpps- 

M@l  dMdl«m  pravidfd  qusHy  pnqdU^q  Id  (i«JqiTiw 

Vol-untfeer  E xpeirierice  and  PeMonal  Intereata 


ThH  0iqiBrie4(iDn  (Vii  Credtwr  IKilcn.  9X3'  Pic^tnl 
■IkiiiJaar 

.  m  flpf ^  jii  oiint  iMrtng  pqitjl  Igr  giKHtp  Iq  ituM-  irutf an  jn4  prqflqtti 


Dynamic  Trust  Model  Project  - 
Overall  Objectives 

•  Explore  the  relative  impact  of  the  trustworthiness  indicators  in  the 
dynamic  trust  model  [Mayer,  1995] 

•  Three  factors  in  the  dynamic  trust  model: 

-  Ability  (competence),  Benevolence  (goodness)  and  Integrity 
(reliability/predictability) 

•  Strength/contribution  of  each  aspect  of  the  trust  Judgment  may  vary: 

-  By  the  type  of  activity  being  performed  e.g.  depending  on  the  role  the  trustee  is 
to  take  (friend,  nanny,  or  boss) 

-  CL  in  context  -  how  mentally  demanding  the  other  tasks  are  while  you  are 
making  the  judgment 

-  By  culture  e.g.  individualistic  vs.  collectivistic  cultural  influences  -  competence 
vs.  benevolence 
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Conceptual  Model  of  Trust 

[Lee  &  See, 2002;  Colquitt  et  al.,  2007;  with  additions  by  Stokes, 

2010] 


Inputs 


Beliefs  &  Perceptions 


Attitudes  Intention 


Behavior 
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•  Trust  is  “willingness  of  an  individual  to  accept  vulnerabilities  from 
others”  [Mayer95] 

•  CL  is  “a  multidimensional  construct  that  represents  the  load  that 
performing  a  particular  task  imposes  on  the  cognitive  system” 
[Paas94]. 

•  Central  to  the  importance  of  CL  is  the  working  memory. 

•  Working  memory  is  defined  as  a  ''limited  capacity  system,  which 
temporarily  stores  information,  and  supports  human  thought 
processes  by  providing  an  interface  between  perception,  long-term 
memory  and  action”  [Baddelley2003]. 

•  Increased  CL  leads  to  overload  of  the  working  memory  resources 
[Baddelley2001]  and  change  in  the  established  cognitive  behaviour 
[Oviatt2004] 
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Hypotheses  and  Expected 
Outcomes 

•  Cognitive  processes  are  invoived  in  cognitive  trust  and  affective  trust 

•  CL  manipuiations  are  iikeiy  to  affect  the  thought  process 

•  CL  manipuiations  are  more  iikeiy  to  affect/disrupt  cognitive  trust 

-  e.g.  gauging  the  suitability  of  a  job  candidate  for  the  manager  role  or  as  a 
colleague 

•  Load  manipuiations  are  iess  iikeiy  to  affect  judgement  whether  a  person  is 
trustworthy  enough  to  be  a  friend 

•  Expect  to  find  greatest  difference  in  experienced  CL  in  decisions  invoiving 
manager/coiieague  than  friend  * 

•  Cuiturai  factors  can  affect  the  interdependence  of  CL  and  trust,  such  that 
cuiturai  biases  in  trust  wiii  be  exacerbated  under  high  CL. 
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User  Study  Design 

Tool 

•  HR  Applicant  Screening  Tool 

-  Developed  in-house,  cross-platform  custom  application  that  incorporates 
all  data  collection,  Dual  versions  (high  CL  and  low  CL) 

-  Several  design,  story  boarding,  wireframe  iterations 


f  a^licajit  68-K  If  "  Biiplicant  59-L  [ 


■■"’licant  47-S 


jj 


Applicant  68-K 

Manager  at  Dlaw  Lighting  and 
Conf igur at ion ,  Ihc . 


Education 


Experience 


Jackson  University  of  Liberai  Arts,  1995 

Bachelor  of  Arts:  Business 


"Acts  on  the  up  and  up  from  what  I've  seen  and  heard. 
Always  fair  and  honest."  -  Professor  of  Business  Finance 


Diaw  Lighting  and  Configuration,  Inc.,  2001  -  Present 
Project  Manager  /  Consultant 

•  Responsibilities:  Job  duties  involved  leading  a  team  of  sales  associates 
and  developing  team  cohesiveness. 

"Has  a  strong  moral  foundation.  Always  does  the  right  thing  even  in 
\r  morally  difficult  situations."  -  Executive  Supervisor 

Boris,  Blackwood,  &  Harris,  Inc.,  1995  -  2001 
Hurrran  Services  Manager 

•  Responsibilities;  Organization  and  development  of  product  development 


Volunteer  Experience  and  Personal  Interests 

eekend  Readers  Book  Club,  2001  -  Present 
smber 

Assisted  in  the  organization  and  planning  of  book  discussions  for  club  members. 

^  "Is  honest,  just,  and  impartial  without  fail."  -  Co-Club  Member 
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User  Study  Design  and 

Data 


•  Study  Design 

-  Tasks:  Rating,  Positioning,  and  Ranking  job  applicants 

-  CL  (High  vs  Low):  dual  task  (queuing  job  applicants) 

-  Manipulation  of  trustee:  my  manager,  my  colleague,  others’  manager 

-  Manipulation  of  trust  factors:  ability,  benevolence,  integrity 

-  Cultural:  Malaysian,  Australian,  American  (collectivistic,  individualistic) 

•  Behavioural  Measures 

-  Speech:  think-aloud  protocol  was  used  for  CLM 

-  Interactive  Behaviour: 

•  mouse  trajectories,  selection,  typing,  browsing  activity  (attentional  focus) 
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User  Study  Design  and 
Data  Continued... 


•  Performance  Measures 

-  Time-to-completion;  time  to  response;  amount  of  speech 

-  Performance  on  secondary  task  (queuing  incoming  applicants) 

-  Trust  ratings  for  different  positions/roles 

-  Rank  ordering  of  the  applicants  into  three  categories/roles 

•  Seif  Report  Measures 

-  Subjective  ratings  of  mental  effort  (Lickert  scales) 

-  Trust  propensity 
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Project  Progress  Overview 


•  Internal  Review  Board  (IRB)  Approvals 

-  All  sites  were  approved. 

•  Pilots 


-  Pre-pilot  material  and  experiment  tool  design  completed 

-  Pilot  experiments  were  conducted  in  mid-2011 

-  Content/manipulations  were  tested  for  bias  using  Survey  Monkey. 

-  A  student  group  from  University  of  New  South  Wales  (Australia)  (UNSW)  (30) 
participated 


•  Wright-Patterson  team  visited  Sydney  late  August  2011 

-  Trial  runs,  pilots  and  think-aloud  tests,  final  interface  designs 

-  Planning  for  data  collection,  analyses  and  next  stage  proposal. 

•  Study/Participants 

-  Australian  group:  90  students  from  the  University  of  Sydney  participated  in  the 
user  study  in  November  2011 

-  Malaysian  group:  130  students  from  Sunway  University  participated  in  the  user 
study  in  July  2012 

-  US  group:  data  collection  in  progress 
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Data  Collection  and  Analysis 

Summary 


•  Australian  data  collection  brief  stats: 

-  90  subjects  completed  both  conditions  (high  and  low  CL) 

-  239  survey/response  data  points  per  subject 

-  Speech  data:  6.5Gb  =  58  hours  of  speech 

-  Interactive  Behaviour:  ~96  million  data  points 

•  including  mouse  trajectories,  selection,  typing,  browsing  activity  (attentional  focus) 

-  Speech  transcriptions:  -80,000  words  from  effective  speech 
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Data  Collection  and  Analysis 
Summary  Continued.... 

•  Analysis  of  Data  from  Australian  site: 

-  Subjective  Analysis  of  CL 

-  Performance  Analysis 

•  Completion  times 

•  Response  times 

•  Amount  of  speech  (proportional  speech,  word  count,  words  per  sentence) 

•  Dual-task  performance 

-  Behavioural  Analysis 

•  Linguistic  data 

•  Speech  data 

•  Mouse  and  interaction  behaviour 
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Analysis  Summary 


•  Analysis  of  Data  from  Australian  site: 

-  Subjective  Analysis  of  CL  (to  validate  the  experiment  design) 

-  Collected  at  the  end  of  both  the  high  load  and  low  load  task  sessions 

-  Based  on  a  7-point  Likert  scale  (1=“Extremely  easy”  and  7=“Extremely  difficult”). 

-  Mean  ratings: 

•  Low  load:  3.037 

•  High  load:  3.625 

•  (f(72)=5.201,p<.001) 
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Analysis  Summary 
Continued . 

•  Analysis  of  Data  from  Australian  site: 

-  Performance  Analysis 

-  Completion  times  and  Response  times 

-  On  average,  participants  spent  more  time  on  high  CL  task  than  low  CL  task 

-  On  average,  their  response  latency  was  higher  under  high  CL  task 

-  Effective  speech  amount 

-  Proportional  speech  -  no  difference  under  low  and  high  CL  (p=0.2) 

-  Average  word  count  -  no  difference  (p=0.1 )  (confirming  above) 

-  Average  #  words  per  sentence  -  longer  sentences  under  high  CL  task 
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Analysis  Summary 
Continued . 


•  Analysis  of  Data  from  Australian  site: 

-  Behavioural  Analysis 

-  Linguistic  Analysis  of  Speech  Data 

•  Over  70  participants’  speech  being  transcribed  and  annotated 

•  Preliminary  analysis  results  based  on  55  subjects 

-  Pause  Analysis 

-  Linguistic  Category  Analysis 
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Analysis  Summary 
Continued . 


•  Analysis  of  Data  from  Australian  site: 

-  Behavioural  Analysis 

•  Pause  Analysis 

•  On  average,  participants  paused  more  under  high  CL  task 

•  On  average,  they  also  paused  longer  under  high  CL  task  than  low  CL  task 


P<0.01 


Avg  #  of  pauses  (min) 

p<0.01 


1.6 
1.4 
1.2 
1 

0.8 
0.6 
0.4 
0.2 
0 

Low  High 


Avg  Length  of  Pauses 

p<0.0001 


%  of  Time  Pausing 

p<0.0001 
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Analysis  Summary 
Continued . 

•  Analysis  of  Data  from  Australian  site: 

-  Behavioural  Analysis 

•  Linguistic  Category  Analysis  (based  on  Linguistic  Inquiry  and  Word  Count  (LIWC) 


Linguistic  Categories 

Example  words 

Personal  pronouns 

1,  they,  her,  we 

Impersonal  pronouns 

it,  those,  it's,  that 

Adverbs 

very,  really,  quickly,  mostly 

Negations 

no,  not,  never,  neither 

Quantifiers 

few,  many,  much,  fairly 

Swear  words 

damn,  shit,  fuck,  piss 

Affective  (emotional)  processes 

happy,  cry,  glad,  afraid 

Positive  emotions 

nice,  sweet,  cool 

Negative  emotions 

ugly,  nasty,  bad,  fail,  sorry 

Anxiety 

worried,  fearful,  nervous 

Anger 

hate,  kill,  annoyed 

Sadness 

sad,  grief,  cry 

Cognitive  processes 

know,  cause,  opinion 

Insight 

think,  know,  consider 

Causation 

hence,  effect,  because 

Discrepancy 

should,  would,  could 

Tentative 

maybe,  perhaps,  guess 

Certainty 

always,  never,  absolutely 

Achievement 

win,  hero,  ability,  perform 

Assent 

agree,  ok,  yes,  cool 

Trust 

trust,  believe,  sure 

Distrust 

doubt,  disbelieve,  suspicious 
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Analysis  Summary 
Continued . 


Analysis  of  Data  from  Australian  site: 

-  Behavioural  Analysis 

•  Linguistic  Category  Analysis  (based  on  LIWC  dictionary) 


%  of  Certainty  Words 


%  of  Achievement  Words 


%  of  Trust  Words 


%  of  Distrust  Words 
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Analysis  Summary 
Continued . 


•  Analysis  of  Data  from  Australian  site: 

-  Behavioural  Analysis 


•  Signal  Analysis  of  Speech  data 

o  Data  cleaning  (e.g.  remove  cross-talk,  segmentation)  completed 
o  Initial  CL  Models  were  built 
o  Initial  analysis  completed 

o  Automatic  classification  of  CL  levels  showed  63%  performance 
o  Currently  working  to  improve  the  results. 

•  Mouse  and  interaction  behaviour  analysis 

o  Mouse  interaction  data  cleaning  completed 
o  Mouse  interaction  features  identified  for  analysis 

o  Currently  in  the  process  of  developing  Machine  Learning  (ML)  method  to  classify  CL 
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Next  Tasks 

•  Finalize  Linguistic  analysis 

•  Speech  analysis  -  refine  CL  models 

•  Interaction  analysis 

-  ML  method  to  be  refined  to  analyse  mouse  interaction  data 

-  Selection  and  Typing  data  analysis 

•  Performance  analysis 

-  Analysis  of  rating/ranking  performance 

-  Dual-task  performance 

•  Relationship  between  CL  and  individual  Trustworthiness  (ABI)  judgements 

•  Intercultural  analysis 

-  Comparison  of  data  collected  from  all  three  sites  (USA,  Malaysia,  Australia) 

•  Consolidating  findings  from  the  three  sites 

•  Write-up  of  findings 

•  Publication  of  findings 

•  Final  Year  Report 

40 

Distribution  A.  Approved  for  public  release;  distribution  unlimited. 

88ABW-2014-1 127,  cleared  20  March  2014 


QUESTIONS? 


CONTACT: 

CHARLENE.STOKES@WPAFB.AF.MIL 
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LIST  OF  ACRONYMS 


ABl 

Ability  Benevolence,  Integrity 

AFRL 

Air  Force  Research  Laboratory 

AOARD 

Asian  Office  of  Aerospace  Research  and 
Development 

Cl 

Cognitive  Load 

IRB 

Internal  Review  Board 

LIWC 

Linguistic  Inquiry  and  Word  Count 

ML 

Machine  Learning 

NICTA 

National  Information  and  Communications 
Technology  Australia 

PI 

Principal  Investigator 

UNSW 

University  of  New  South  Wales 
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